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EXECUTIVE  SUMMARY 


OBJECTIVE 

The  Tactical  Decision  Making  Under  Stress  (TADMUS)  program  is  being  conducted  to  apply 
recent  developments  in  cognitive  theory  and  human-system  interaction  technology  to  the  design  of  a 
decision  support  system  (DSS)  for  enhancing  tactical  decision-making  under  the  highly  complex 
conditions  involved  in  littoral  settings  or  any  short-fused,  dynamic  decision-making  situation.  Our 
goal  is  to  present  decision  support  information  in  a  format  that  (1)  minimizes  any  mismatches 
between  the  cognitive  characteristics  of  the  human  decision  maker  and  the  design  and  response  char¬ 
acteristics  of  the  decision  support  system,  (2)  mitigates  the  shortcomings  of  current  tactical  displays 
that  impose  high  information  processing  demands  and  exceed  the  limitations  of  human  memory,  and 
(3)  synthesizes  numeric  data  into  graphic  representations  to  facilitate  the  interpretation  of  spatial 
data. 

RESULTS 

The  research  reported  here  focuses  on  developing  a  DSS  that  reflects  the  natural  decision-making 
strategies  of  humans.  Hence,  prototype  display  development  was  based  on  decision-making  models 
postulated  by  naturalistic  decision-making  theory,  such  as  the  recognition-primed  decision  model  and 
explanation-based  reasoning.  In  addition,  human-system  interaction  design  principles  have  been 
incorporated  which  are  expected  to  reduce  cognitive-processing  demands  and  thereby  mitigate  deci¬ 
sion  errors  caused  by  cognitive  overload,  which  have  been  documented  through  research  and  exper¬ 
imentation.  Presenting  synthesized  information  in  the  form  of  graphic  presentations  is  expected  to 
reduce  the  cognitive-processing  load  for  the  decision  maker  when  performing  situation  assessment. 
The  intention  is  to  aid  the  decision  maker  by  providing  information  in  a  way  that  will  minimize  the 
need  to  maintain  information  in  working  memory,  reduce  information-processing  demands,  help 
focus  attentional  resources  on  the  highest  priority  contacts,  remind  the  user  of  actions  that  must  be 
taken,  help  make  decisions  under  stress,  and  support  higher  levels  of  situation  awareness. 

Features  offered  by  the  DSS  to  address  errors  attributed  to  limited  attentional  resources  include 
focusing  attention  on  (1)  high  priority  contacts  (i.e.,  track  priority  list  and  alerts),  as  well  as  on 
(2)  missing  data  (e.g.,  threat  assessment),  and  (3)  enabling  the  decision  maker  to  use  more  data  than 
are  typically  used  in  current  systems  (such  as,  track  history  and  comparison  to  normal  values).  Cur¬ 
rent  systems  require  the  user  to  retain  previous  contact  data  in  memory  to  compare  with  the  most 
recent  values  for  critical  parameters.  Current  systems  also  require  the  user  to  rely  on  recall  of  vast 
amounts  of  information  from  training  and  experience.  Presenting  all  known  data  on  a  contact  in  a 
synthesized  way  should  reduce  working-memory  requirements  and  facilitate  recognition.  Additional 
potential  performance  enhancement  features,  offered  by  the  DSS,  include  displaying  the  complete 
kinematic  contact  history,  presenting  graphic  displays  of  location  and  trends,  highlighting  missing 
data,  providing  alerts,  and  providing  assessments  of  current  contact  identity  that  go  beyond  what 
existing  systems  now  present. 

RECOMMENDATIONS 

Many  of  the  recommendations  made  by  subjects  in  the  initial  evaluation  of  the  DSS  have  been 
incorporated  along  with  additional  features  the  research  team  anticipates  will  further  enhance  perfor¬ 
mance.  The  next  phase  of  the  TADMUS  program  will  empirically  evaluate  the  DSS  II.  Other 
research  issues  remain,  which  include  plans  for  (1)  testing  the  complementary  roles  of  the  DSS  and 


(2)  training  in  critical  thinking.  Critical  thinking  is  a  form  of  critiquing  the  currently  held  threat 
assessment  to  ensure  one  has  not  succumbed  to  cognitive  biases.  An  experiment  will  be  conducted  to 
examine  whether  practice  and  feedback  provided  by  the  DSS  (using  a  modified  version  of  the  threat 
assessment  module)  enhances  training. 
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INTRODUCTION 


Recent  changes  in  U.S.  Naval  priorities  stress  the  requirement  for  Navy  ships  to  operate  in  the  litto¬ 
ral  regions  of  the  world,  that  is,  in  the  coastline  regions.  Operating  in  the  congested  and  confined 
water  and  airspace  close  to  land  presents  additional  challenges  to  the  tactical  decision  maker.  Littoral 
operations  involve  scenarios  characterized  by  rapidly  unfolding  events  that  fit  multiple  possible 
hypotheses  concerning  contact  identification,  intent,  available  responses,  and  their  consequences.  For 
example,  the  close  proximity  of  U.S.  Navy  forces  and  potential  adversary  forces  greatly  complicates 
interpreting  the  actions  of  an  inbound  aircraft  that  does  not  respond  to  radio  warnings.  Should  the  air¬ 
craft’s  behavior  be  interpreted  as  an  attack  profile?  Or  does  the  pilot  merely  intend  to  harass?  Or  does 
the  aircraft  in  question  not  carry  the  equipment  needed  to  receive  verbal  warnings,  leaving  the  pilot 
unable  to  receive  radio  warnings  directed  toward  him  and  unaware  of  his  precarious  position?  In 
extreme  cases,  there  is  no  clear-cut  right  or  wrong  answer  about  a  decision.  Rapidly  unfolding  events 
result  in  severe  time  pressure  and  severe  (often  catastrophic)  consequences  for  errors.  While  current 
real-time  battle-management  systems  are  well-suited  to  the  demands  of  all-out  conflicts,  they  may  not 
be  optimized  for  littoral  situations  where  human  intervention  in  decision-making  is  even  more  impor¬ 
tant  (Office  of  Naval  Technology  [ONT],  1992).  (Since  70  percent  of  the  world’s  population  lives 
within  200  miles  of  the  sea,  most  future  contingencies  are  likely  to  involve  littoral  warfare  (Mundy 
1994).) 

Two  unfortunate  and  highly  publicized  events  focused  attention  on  the  difficult  types  of  decisions 
confronting  naval  commanders  and  provided  the  impetus  for  this  research.  In  the  case  of  the 
USS  Stark,  the  conunander  made  the  decision  not  to  engage  an  inbound  aircraft  that  was  not  believed 
to  be  a  threat  to  his  ship,  resulting  in  27  U.S.  Naval  personnel  losing  their  lives.  In  the  case  of  the 
USS  Vincennes,  the  commander  made  the  decision  to  engage  the  inbound  aircraft,  believing  it  was  a 
threat  to  his  ship — which  turned  out  not  to  be  a  threat,  but  a  commercial  airliner— resulting  in  all  per¬ 
sonnel  aboard  the  airliner  being  killed.  In  recognition  of  the  complex  and  difficult  decisions  required 
in  these  types  of  situations,  the  Tactical  Decision  Making  Under  Stress  (TADMUS)  program  was  ini¬ 
tiated  to  conduct  research  in  human  factors  and  training  technology.  The  objective  is  to  develop  and 
apply  principles  that  can  help  avoid  these  types  of  simations  in  the  future.  This  document  reports  on 
research  conducted  under  the  TADMUS  program  to  apply  recent  developments  in  decision  theory 
and  human-system  interaction  technology  to  the  design  of  a  decision  support  system  (DSS)  for 
enhancing  tactical  decision-making  under  highly  complex  conditions. 

“NATURALISTIC”  AND  CLASSICAL  DECISION-MAKING  PARADIGMS 

In  the  same  time  frame  that  these  tragic  accidents  occurred,  a  radical  shift  was  occurring  in  the  way 
psychologists  viewed  human  decision-making.  Research  was  now  focused  on  experienced  decision 
makers  performing  their  normal  tasks  in  natural  settings.  “Naturalistic”  decision-making  research 
studies  the  decision  strategies  people  actually  use  in  bringing  their  expertise  to  bear  under  challenging 
real-world  conditions.  These  conditions  include  dynamic,  fluid  situations;  time  pressure;  high-risk, 
multiple  decision  makers;  shifting  and  competing  goals;  action-feedback  loops;  and  situations  with 
uncertain  and  incomplete  data  (Orasanu  &  Connolly,  1993).  Decision-making  milieus  encompassed 
under  the  naturalistic  paradigm  include  hospital  emergency  rooms,  aircrew  flight  coordination,  mili¬ 
tary  command  and  control  settings,  process  control  systems,  and  police  and  fire  units. 

The  naturalistic  perspective,  also  known  as  “everyday  cognition,”  is  based  on  a  belief  that  cognitive 
functions  “elicited  in  natural  settings  (are)  likely  to  differ,  either  quantitatively  or  qualitatively  from 
those  that  occur  in  artificial  or  contrived  situations;  and  results  from  sterile  and  contrived  situations 
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may  not  generalize  to  less  constrained  and  more  natural  environments”  (Salthouse,  1992,  p.  982). 
There  is  a  growing  body  of  work  that  demonstrates  that  experienced,  real-world  decision  makers 
rarely  use  traditional  resource-intensive  strategies  to  make  decisions  in  the  face  of  dynamic,  adverse 
conditions  and  time-pressure  (Kaempf  &  Militelo,  1992;  Klein,  1993).  Instead,  experts  rely  on  their 
abilities  to  recognize  and  appropriately  classify  situations — ^these  abilities  are  based  on  having  much 
experience  in  the  task  domain.  Once  these  experts  know  what  they  are  facing,  they  also  tend  to  know 
what  response  option  to  apply,  based  on  retrieving  from  memory  typical  responses  and  outcomes  that 
worked  well  in  past  similar  situations.  They  use  the  limited  time  available  to  evaluate  the  feasibility 
of  that  option  before  implementing  it.  Experienced  decision  makers  recognize  the  situation  or  sce¬ 
nario  based  on  comparing  the  features  of  the  current  situation  with  stored  memory  representations,  or 
schemata.  Schemata  are  highly  interconnected  clusters  of  knowledge  concerning  certain  situations,  or 
particular  problem  types,  and  associated  actions  or  solution  procedures  (Federico,  1995).  Once  the 
situation  is  recognized,  solutions  are  stimulated  by  activating  these  memory  representations. 

In  contrast  to  the  naturalistic  perspective,  earlier  analytical  methods  applied  in  decision  support 
systems  were  primarily  used  for  option  generation  and  evaluation,  rather  than  for  situation  assess¬ 
ment.  Traditional  decision  theorists  argue  that  optimum  decision-making  involves  thorough  analysis 
of  all  the  available  data  and  evaluation  of  all  possible  hypotheses;  these  approaches  tend  to  rely  on 
extensive  calculations  designed  to  arrive  at  optimal  solutions.  Processes  for  making  decisions  where 
the  person  weighs  the  pros  and  cons  of  various  options  and  selects  the  option  that  provides  the  most 
benefit  are  described  in  the  decision-making  literature.  The  extensive  time  requirements  and  compli¬ 
cated  mathematical  calculations  involved  (e.g..  Multiattribute  Utility  Analysis),  however,  are  unreal¬ 
istic  for  situations  requiring  rapid  decision-making.  Generally,  these  analytical  strategies  involve  the 
following  steps  (Kaempf  &  Militelo,  1992): 

•  Specify  all  relevant  features  of  the  task, 

•  Identify  the  full  range  of  options, 

•  Identify  the  key  evaluation  dimensions, 

•  Identify  weights  for  each  dimension, 

•  Rate  each  option  on  each  dimension, 

•  Tabulate  the  results,  and 

•  Select  the  best  option. 

These  analytical  strategies  may  be  appropriate  for  inexperienced  subjects  making  decisions  about 
novel  tasks,  but  not  for  experienced  personnel  making  real-time  decisions.  In  natural  settings,  time 
constraints  and  the  difficulty  in  assigning  weights  and  rating  the  dimensions  involved  render  classical 
analysis  techniques  untenable.  Typically,  in  realistic  settings,  experts  employ  recognition-based  rea¬ 
soning,  not  classical  analytical  approaches.  Experienced  decision  makers  use  their  extensive  knowl¬ 
edge  to  seek  information,  identify  and  interpret  the  problem,  understand  the  significance,  derive  the 
intention  (where  possible),  model  the  situation  (as  time  allows),  select  the  action,  evaluate  the  choice, 
and  anticipate  the  consequence.  “This  decision  cycle  is  distinctly  different  from  classical  models, 
which  are  based  on  the  assumption  that  all  options,  outcomes,  and  preferences  are  known  and  calcu¬ 
lated  in  advance”  (Federico,  1995,  p.  106). 

Traditionally,  research  in  decision-making  has  been  directed  largely  toward  situations  in  which 
(1)  decision-makers  have  sufficient  time  to  generate  options,  conduct  option  assessment,  and  select  a 
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course  of  action;  (2)  the  consequences  of  an  incorrect  response  are  not  immediately  severe;  (3)  deci¬ 
sions  are  reached  via  consensus  of  a  group;  and  (4)  workload  is  manageable.  Little  research  has  been 
conducted  into  development  of  tactical  decision  support  systems  for  use  in  naturalistic  situations 
characterized  by  time  pressure,  high  risk,  uncertainty  and  information  ambiguity,  high  workload,  team 
coordination  demands,  and  task  complexity  (ONT,  1992). 

The  central  hypothesis  for  the  research  reported  here  is  that  presenting  decision  makers  with  deci¬ 
sion  support  tools  designed  to  parallel  the  cognitive  strategies  employed  by  experts,  as  observed  in 
naturalistic  settings,  will  reduce  the  number  of  decision-making  errors.  This  is  accomplished  by 
developing  the  architecture  and  algorithms  to  process  information  the  same  way  research  indicates 
humans  do  under  similar  circumstances. 
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DECISION  SUPPORT  PRINCIPLES 


A  case  has  been  made  that  previous  generations  of  decision  support  systems  are  less  applicable 
than  a  decision  support  system  (DSS)  that  parallels  the  cognitive  strategies  used  by  domain  experts  in 
situations  characterized  by  time-constrained  circumstances  with  uncertain  and  ambiguous  data 
(Smith  &  Grossman,  1993).  Such  systems  focus  primarily  on  solution  optimization  and  base  decision 
support  on  normative  models  of  human  decision-making.  These  authors  point  out  that  rarely,  if  ever, 
were  earlier  tactical  decision  aids  intended  as  psychological  models  of  human  cognitive  behavior. 
Instead,  these  aids  performed  “complex  and  burdensome  calculations,  reducing  the  workload  on  per¬ 
sonnel,  speeding  up  the  dissemination  of  information,  and  providing  more  time  for  command  deci¬ 
sion-making”  (Tolcott,  1991,  p.  44). 

FEATURE  MATCHING  AND  STORY  GENERATION 

We  have  applied  two  of  these  new  models  of  human  decision-making — ^which  parallel  the  cogni¬ 
tive  strategies  used  by  domain  experts — to  the  design  of  a  DSS  for  enhancing  antiair  warfare  tactical 
decision-making.  These  two  models  that  people  use  in  assessing  a  situation  are  feature  matching  and 
story  generation.  The  feature-matching  model,  described  by  Noble  (1989),  involves  an  organization 
of  memory,  or  “schemas,”  and  information  processing  where  decision  makers  use  their  previous 
experiences  to  assess  a  situation  and  identify  promising  actions.  Incoming  information  is  categorized, 
selected,  edited,  and  organized  on  the  basis  of  a  person’s  general  knowledge  about  a  domain.  Both 
story  generation  and  feature  matching  occur  under  conditions  where  a  large  base  of  implication-rich, 
conditionally  dependent  pieces  of  evidence  must  be  evaluated  before  choosing  an  alternative  from  a 
set  of  prospective  courses  of  action.  The  feature-matching  model  applies  a  spatio-temporal  depen¬ 
dence,  whereas  story  generation  is  an  example  of  causal  dependence.  According  to  the  explanation- 
based  model,  decision  makers  construct  a  causal  model  to  explain  the  available  evidence  (Pennington 
&  Hastie,  1993).  At  the  same  time,  the  decision  maker  creates  a  set  of  alternatives  from  which  an 
action  will  be  chosen.  A  decision  is  made  when  a  story  is  successfully  matched  to  an  alternative  in 
the  choice  set.  Story  generation  occurs  in  complex  situations  where  the  decision  maker  may  not  have 
all  the  necessary  information  or  when  a  series  of  facts  may  appear  to  contradict  each  other.  The  deci¬ 
sion  maker  must  then  develop  causal  links  between  these  facts  to  produce  a  coherent  picture  of  the 
situation  (Klein,  1989;  1993). 

The  story  generation  model  is  based  on  research  which  found  that  jurors  develop  a  narrative  story 
to  organize  trial  information  where  causal  and  intentional  relations  between  events  are  central  (Pen¬ 
nington  &  Hastie,  1992).  Pennington  &  Hastie  propose  four  certainty  principles — coverage,  coher¬ 
ence,  uniqueness,  and  goodness-of-fit — that  govern  which  story  will  be  accepted,  which  decision  will 
be  selected,  and  the  confidence  or  degree  of  certainty  with  which  a  particular  decision  will  be  made. 
This  organization  of  the  evidence  by  the  decision  maker  is  believed  to  facilitate  evidence  comprehen¬ 
sion.  A  central  component  of  this  model  is  that  the  story  the  juror  constructs  determines  the  juror’s 
decision.  This  explanation-based  decision  process  is  employed  when  the  body  of  evidence  relevant 
to  a  decision  is  large,  complex,  and  the  implications  of  its  components  are  interdependent. 

Feature  matching,  also  referred  to  as  the  recognition-primed  decision  (RPD)  model,  “occurs  when 
the  decision  maker  recognizes  the  features  of  the  present  situation  as  similar  or  identical  to  those  of  a 
previous  situation”  (Kaempf  &  Militelo,  1992,  p.  6).  An  adequate  match  triggers  recall  of  information 
learned  about  this  type  of  situation;  that  is,  plausible  goals,  critical  cues  to  be  monitored,  expectations 
of  what  should  happen,  and  a  course  of  action  that  worked  in  similar  simations.  According  to  this 
recent  approach,  expert  decision  makers  may  rely  on  well-developed  memory  representations  to 
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guide  decision-making  in  new  (but  similar)  situations.  The  RPD  model  of  decision-making  fuses  two 
processes — situation  assessment  and  mental  simulation  (Klein,  1993).  In  the  simplest  case,  the  situa¬ 
tion  is  recognized  as  familiar  or  prototypical,  using  feature  matching,  and  the  obvious  response  is 
implemented.  In  a  more  complex  case,  the  decision  maker  consciously  evaluates  the  response,  using 
mental  simulation  to  uncover  problems  prior  to  implementing  the  response.  In  the  most  complex  case, 
the  evaluation  reveals  flaws  requiring  modification,  or  the  option  is  judged  inadequate  and  rejected  in 
favor  of  the  next  most  typical  reaction. 

Situation  Assessment 

In  general,  the  overall  task  of  responding  to  antiair-warfare  (AAW)  scenarios  consists  of  situation 
assessment  (“what’s  going  on”)  and  course-of-action  selection  (“what  to  do  about  it”).  Recent  theo¬ 
ries  of  decision-making  emphasize  the  importance  of  situation  assessment  for  good  decision-making 
in  naturalistic,  event-driven  situations.  Moreover,  they  stress  that  decisions  regarding  actions  to  be 
taken  are  a  by-product  of  developing  the  situation  awareness  that  precedes  action  selection.  Klein 
(1989)  has  found  that  usually  the  situation  itself  either  determines  or  constrains  the  response  options 
and  that  experienced  decision  makers  make  up  to  90  percent  of  all  decisions  without  considering 
alternatives.  If  the  situation  appears  similar  to  one  the  decision  maker  has  previously  experienced,  the 
pattern  will  be  recognized,  and  the  course  of  action  is  usually  immediately  obvious.  On  the  other 
hand,  if  the  situation  does  not  seem  familiar,  complex  RPD  will  be  involved,  where  the  decision 
maker  adjusts  the  option  after  evaluating  it. 

Additional  evidence  was  found  in  the  specific  task  domain  of  interest  to  the  TADMUS  program, 
which  added  support  to  these  findings  on  the  way  real-world  decision  makers  make  decisions  in  the 
context  of  their  normal  jobs.  Research  was  conducted  to  determine  decision  requirements  for 
command-level  decision  makers  in  the  combat  information  center  (CIC)  of  an  Aegis  cruiser.  Analysis 
of  14  incidents  from  actual  problems  revealed  1 83  decisions.  Of  these,  103  concerned  situation 
assessments.  Results  obtained  after  analysts  coded  these  situation  assessments  indicated  that  decision 
makers  arrived  at  87  percent  of  their  situation  assessments  through  feature  matching  and  the  remain¬ 
ing  13  percent  through  story  generation  (Kaempf,  Wolf,  &  Miller,  1993).  The  other  80  decisions  that 
were  identified  from  analysis  of  the  real-world  incidents  mentioned  above  involved  course-of-action 
selection.  These  course-of-action  decisions  served  a  variety  of  functions,  although  relatively  few  were 
intended  to  end  the  incident.  Of  these,  20  were  intended  as  a  final  course-of-action  decision,  14  were 
implemented  to  obtain  more  information,  22  to  manage  resources,  and  24  to  put  themselves  in  a  more 
favorable  tactical  position.  A  recognition-based  strategy  was  also  used  by  decision  makers  to  develop 
a  course  of  action,  accounting  for  95  percent  of  the  actions  taken  in  the  14  incidents.  The  decision 
makers  generated  and  compared  multiple  options  in  only  5  percent  of  the  cases.  In  line  with  these 
findings,  the  TADMUS  program  has  adopted  the  position  that  decision-aiding  systems  should  assist 
in  the  decision-making  process  and  focus  on  aiding  the  simation-assessment  portion  of  the  decision¬ 
making  task. 

A  DSS  was  developed  to  support  decision-making  processes  that  research  has  shown  are  used  by 
decision  makers  in  real-world  settings.  Specifically,  the  DSS  parallels  the  strategies  used  by  experi¬ 
enced  decision  makers  to  perform  situation  assessment  (Nobel,  1989;  1993).  This  approach  to  sup¬ 
porting  the  user’s  intuitive  approach  to  dealing  with  dynamic  decision-making  situations  should  pro¬ 
duce  tools  that  are  both  more  easily  understood  and  used,  and  that  more  effectively  “exploit  the 
decision  maker’s  knowledge  and  expertise  which  might  facilitate  adaptation  to  complex,  novel  situa¬ 
tions”  (Cohen,  1993,  p.  265). 
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HUMAN-SYSTEM  INTERACTION  DESIGN  PRINCIPLES 


The  vast  majority  of  research  on  human-computer  interaction  has  been  devoted  to  characteristics  of 
displays  that  impact  human  perception,  such  as  symbol  legibility  or  detectability,  and  on  relatively 
simple  cognitive  functions,  such  as  memory  tasks.  Fewer  efforts  have  been  devoted  to  understanding 
the  effects  of  the  format  and  manner  in  which  information  is  presented  on  more  complex  levels  of 
human  cognition,  for  example,  decision-making.  Consequently,  principles  that  can  be  applied  to  the 
design  of  the  interface  between  the  user  and  a  decision  support  system  for  enhancing  cognitive  pro¬ 
cesses,  particularly  in  stressful,  dynamically  changing  situations,  are  not  available  to  any  significant 
degree  (ONT,  1992). 

GRAPHIC  PRESENTATIONS 

Several  advantages  are  offered  by  graphic  presentations  over  a  text-based  presentation  format  (Lar¬ 
kin  &  Simon,  1987).  Graphic  presentations  should  (1)  reduce  the  amount  of  mental  computation 
required  to  perform  tasks  and  (2)  allow  users  to  spend  less  time  searching  for  needed  information. 
Casner  (1991)  elaborated  on  these  ideas  and  found  that  graphics  allow  users  to  substitute  less 
demanding  perceptual  operations  for  more  complex  logical  operations.  For  example,  determining  a 
change  in  altitude  (and  the  degree  of  change)  is  immediately  apparent  when  the  user  glances  at  the 
track  history  module.  (Note  that  the  words  contact  and  track  can  be  used  interchangeably.  Refer  to 
figure  1  in  the  section  that  describes  the  decision  support  system.)  The  track  history  module  is  located 
in  the  upper  right-hand  comer  of  the  display.  The  objective  for  this  module  is  to  facilitate  the  contact 
identification  process  by  providing  information  that  is  integrated  to  support  a  recognitional  decision 
strategy.  The  track  history  module  depicts  a  contact’s  speed,  altitude,  course,  and  range  on  a  two-di¬ 
mensional  graphical  display  along  with  a  geometric  representation  of  both  the  contact’s  worst-case 
weapon-release  envelope  and  own-ship’s  weapons  coverage.  A  large  amount  of  parametric  data  is 
portrayed  graphically  for  rapid  assimilation  by  the  user.  The  user  can  see,  at  a  glance,  a  synthesized 
picture  of  the  contact’s  behavior.  Compare  this  rather  simple  perceptual  operation  to  the  more  com¬ 
plex  logical  operation  involved  in  current  operational  systems  that  require  the  user  to  recall  and  sub¬ 
tract  numerical  values  for  past  and  current  altitudes. 

Graphics  also  allow  users  to  omit  steps  that  are  otherwise  necessary  when  performing  a  task  with¬ 
out  a  graphic.  An  example  of  this  advantage  is  also  illustrated  in  the  track  history  module,  which 
includes  templates  indicating  weapon’s  coverage  for  both  the  inbound  contact  and  “own-ship.”  To 
determine  whether  the  aircraft  is  within  its  weapon’s  launch  range,  there  is  no  need  to  recall  the  spe¬ 
cific  launch-range  values  and  then  compare  them  with  the  aircraft’s  current  range.  Instead,  the  user 
can  determine  if  the  aircraft  is  within  its  launch  range  by  a  quick  glance  at  the  display. 

Graphics  help  users  save  time  when  searching  for  needed  information  when  several  related  dimen¬ 
sions  of  information  are  encoded  in  a  single  graphical  object.  This  is  accomplished  by  integrating  the 
kinematic  parameters  of  speed,  course,  altitude,  bearing,  and  range  for  a  contact.  The  user  can  see,  at 
a  glance,  a  synthesized  picture  of  the  contact’s  behavior.  Compare  this  process  to  reading,  in  a  text- 
based  format,  the  individual  parameters  that  must  be  integrated  by  the  user  into  a  coherent  picture  of 
the  contact’s  behavior. 
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TACTICAL  DECISION-MAKING  TASKS 


The  global  tactical  decision-making  task  involves  identifying  and  responding  to  numerous  contacts. 
When  an  aircraft  (or  a  surface  contact)  is  detected,  the  CIC  personnel  work  as  a  team  to  determine  its 
identity  and  to  try  to  determine  whether  or  not  the  aircraft  poses  a  threat.  The  high  degree  of  inherent 
ambiguity  associated  with  contact  information  can  often  make  threat  assessment  a  very  difficult  task. 
This  is  because  many  pieces  of  data  fit  multiple  hypotheses  concerning  threat  assessment.  The  global 
response  choices  (that  is,  engage,  monitor,  do  nothing)  are  largely  determined  by  the  ship’s  orders  and 
the  current  geopolitical  situation.  Specific  actions  (such  as,  change  course,  issue  verbal  warnings,  illu¬ 
minate  with  radar,  challenge  with  other  sensors,  etc.)  depend  on  the  local  conditions  and  the  relative 
positions  of  the  inbound  contact  of  interest  and  own-ship.  Determining  which  of  these  actions  is 
likely  to  be  effective  depends  on  maintaining  an  accurate  threat  assessment  which  requires  “continual 
updating  according  to  recurrent  situation  assessments”  (Sarter  &  Woods,  1991,  p.  52). 

This  decision  problem  presents  a  highly  challenging  cognitive  task,  that  is,  making  inferences  and 
deductions  from  incomplete  and  uncertain  information  derived  from  multiple  sources  and  relating  to 
several  concurrent  threats  (or  potential  threats)  under  time-compressed  conditions.  The  cognitive 
functions  performed  by  the  tactical  decision  maker  are  both  data  and  resource  limited  (Norman  & 
Bobrow,  1975).  Decisions  are  resource  limited  by  the  mental  resources  of  the  decision  makers,  who 
must  maintain  large  amounts  of  information  in  memory  under  conditions  of  high  workload  and  stress. 
The  decisions  are  data  limited  by  the  inability  of  the  sensors  to  provide  complete,  error-free,  unam¬ 
biguous  data  to  support  the  identification  process.  In  particular,  the  experimental  scenarios  were 
designed  to  follow  the  pattern  of  being  set  in  an  ambiguous  situation,  where  one  or  more  threats  of 
uncertain  origin  and  uncertain  intent  approach  either  own-ship  or  the  ship  being  protected  and  may 
not  respond  to  warnings.  Scenarios  were  designed  to  be  highly  ambiguous,  as  this  quality  of  uncer¬ 
tainty  is  indicative  of  the  types  of  decisions  to  be  made  in  current  and  future  scenarios. 

THREAT  ASSESSMENT 

In  the  AAW  problem,  threat  assessment  is  particularly  difficult  because  the  available  information  is 
often  incomplete  or  ambiguous.  The  ambiguity  could  be  due  to  (a)  the  information  transmission  char¬ 
acteristics  of  the  transmission  medium,  such  as  a  radar  transmission  or  a  radio  report  that  is  only 
intercepted  on  an  intermittent  basis;  (b)  deliberate  deceptive  actions  (such  as  radar  jamming)  by  the 
pilot  flying  the  aircraft;  or  (c)  the  overlapping  classification  categories  typical  of  many  parameter 
measurements.  For  example,  aircraft  can  typically  fly  at  altitudes  ranging  between  2,000  and 
40,000  feet.  Generally,  an  aircraft  that  is  flying  above  20,000  feet  is  not  considered  to  be  a  threat. 
Conversely,  an  aircraft  below  10,000  feet  is  considered  to  be  more  of  a  potential  threat.  However, 
aircraft  flying  in  the  middle  range,  (that  is,  below  20,000  feet  and  above  10,000  feet)  can  be  much 
more  difficult  to  categorize.  Because  many  aircraft  do  fly  in  this  middle  range,  other  variables  need  to 
be  considered  in  conjunction  with  altitude.  This  same  situation  of  overlapping  categorization  catego¬ 
ries  exists  for  several  other  variables.  These  variables  include  radars  found  on  both  threat  and  non- 
threat  platforms,  country  of  origin,  and  measures  of  course  and  speed.  In  the  case  of  speed,  for  exam¬ 
ple,  when  an  aircraft  flying  at  a  low  altitude  decreases  speed,  this  could  be  viewed  as  indicative  of  a 
threat  action  (that  is,  slowing  down  in  order  to  obtain  better  targeting  information).  However,  at  the 
same  time,  there  could  be  other  viable  explanations  for  an  aircraft’s  decreasing  speed. 

If  the  decision  maker  had  access  to  all  data  about  a  contact,  approximately  12  variables  would  be 
used  to  determine  identity  and  to  infer  intent.  Two  or  three  of  these  items,  alone,  do  not  provide  defin¬ 
itive  answers  because,  in  many  cases,  these  parameter  values  do  not  fall  within  clear-cut  ranges  for  a 
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particular  assessment  category  (i.e.,  threat,  nonthreat).  Thus,  a  single  time  slice  of  information  pro¬ 
vides  an  incomplete  picture  of  the  situation.  In  the  dynamic,  ambiguous  conditions  characteristic  of 
littoral  operations,  the  rate  and  direction  of  change  (data  history)  can  help  one  better  assess  the  threat 
and  predict  the  future  state  of  the  situation.  When  the  incoming  information  changes  over  time,  the 
integration  of  information  as  it  changes  can  help  the  user  extract  the  message  (Kirshenbaum,  1992). 
The  DSS  was  designed  to  do  precisely  this:  to  facilitate  the  integration  process  and  present  a  synthe¬ 
sized  picture  of  the  situation  to  the  user  in  a  format  that  can  be  quickly  assimilated.  The  variables 
used  to  develop  a  threat  assessment  can  be  divided  into  two  classes:  sensor  information  (raw  or  com¬ 
puter-processed  information)  and  the  contact’s  response,  or  lack  of  response,  to  actions  taken  by  the 
team.  These  other  actions,  and  the  integration  of  the  information  received  via  the  contact’s  response 
or  lack  of  response  to  them,  are  needed  to  clarify  the  tactical  picture. 

SITUATION  AWARENESS 

While  recent  increasing  interest  among  researchers  about  the  concept  of  situation  awareness  (SA) 
has  generated  a  debate  on  the  precise  definition  of  this  term,  most  researchers  acknowledge  the 
importance  of  the  concept.  In  general,  SA  refers  to  the  decision  maker’s  moment-by-moment  ability 
to  monitor  and  understand  the  state  of  the  complex  system  and  its  environment  (Adams,  Tenney, 

&  Pew,  1995).  These  authors  state  the  essential  idea,  which  is  that,  when  emergencies  arise,  the  com¬ 
pleteness  and  accuracy  of  the  decision  maker’s  SA  are  critical  to  the  ability  to  make  decisions,  revise 
plans,  and  manage  the  system.  Specific  decision-making  tasks  included  under  SA  include  the  ability 
to  (1)  maintain  an  accurate  perception  of  the  surrounding  environment  (both  internal  and  external  to 
the  ship),  (2)  identify  problems  and/or  potential  problems,  (3)  recognize  a  need  for  action,  (4)  note 
deviations  in  the  mission,  and  (5)  maintain  awareness  of  tasks  performed  (Shrestha,  Prince,  Baker, 

&  Salas,  1995).  To  maintain  an  accurate  SA,  the  deeision  maker  should  take  into  account  both 
information  that  is  available  and  that  which  can  be  activated  from  memory  (Sarter  &  Woods,  1991). 

However,  a  difficulty  arises  as  a  result  of  the  heavy  workload  imposed  by  this  process.  Memory 
load  easily  exceeds  human  capacity  when  the  decision  maker  is  faced  with  several  concurrent  con¬ 
tacts  of  interest,  all  of  which  have  numerous  associated  data  items  (i.e.,  as  many  as  a  dozen),  some  or 
all  of  which  may  change  over  the  course  of  the  scenario  (e.g.,  intelligence,  active  radar  emitters,  vari¬ 
ous  kinematic  parameters,  etc.).  Moreover,  changing  parameters  may  impart  different  interpretations 
to  what  is  occurring.  In  some  cases,  the  moment-to-moment  attentional  demands  of  a  tactical  situation 
are  relentless  and  unforgiving  (such  as  a  terrorist  aircraft  directly  inbound  toward  “own-ship”  that  can 
result  in  “task  fixation”).  Sometimes  relevant  background  knowledge  is  unavoidably  incomplete 
(such  as  an  unfamiliar  aircraft);  and  sometimes  the  decision  maker  is  already  thinking  and  working  as 
hard  as  possible,  even  when  there  are  no  unanticipated  events  during  a  high-contact  density  (Adams 
et  al.,  1995).  These  instances  provide  a  few  illustrations  of  situations  that  can  challenge  situation 
awareness. 

Complex  information-gathering  and  processing  systems  have  been  designed  to  aid  the  decision 
maker  in  the  past.  However,  these  systems  often  increase  the  decision  maker’s  burden  due  to  the 
inherent  system  complexity  and  the  failure  to  design  them  so  they  will  fit  the  user’s  cognitive- 
processing  limitations.  Often,  these  systems  require  operators  to  perform  difficult  cognitive  tasks 
under  heavy  workloads.  They  must  perceive,  synthesize,  and  determine  the  relevance  of  a  continual 
stream  of  incoming  information,  often  pertaining  to  several  concurrent  contacts,  while  projecting 
future  anticipated  events  and  making  decisions  about  actions  to  be  taken.  Decision  makers  must 
assess,  compare,  and  resolve  conflicting  information,  while  making  difficult  judgments  and  remem¬ 
bering  the  status  of  critical  contacts  along  with  the  contact’s  response  to  actions  taken  by  the  CIC 
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team.  These  decision-making  tasks  are  interleaved  with  other  required  tasks,  such  as  keeping  other 
team  members  informed  (both  on  and  off  the  ship).  Furthermore,  these  complex  tasks  are  performed 
under  conditions  where  adverse  environmental  (noise,  vibration,  temperature  extremes,  etc.)  and 
internal  stressors  (boredom,  fatigue,  anxiety,  and  fear)  are  part  of  the  environment. 
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LIMITED  HUMAN  COGNITIVE-PROCESSING  CAPABILITIES 


Since  there  are  limits  to  the  cognitive-processing  capability  of  humans,  the  system  must  provide  the 
needed  information  in  a  format  that  best  supports  the  user  operating  under  dynamic  decision-making 
conditions.  It  may  be  the  case  that  current  systems  are  inadequate  to  support  the  cognitive-processing 
demands  required  by  certain  littoral  scenarios  or  in  any  short-fused,  ambiguous  decision-making  situ¬ 
ation.  For  example,  according  to  Gruner  (1990,  p.  41),  the  USS  Vincennes^  officers  and  system  opera¬ 
tors  “could  not  make  better  decisions  because  they  did  not  have  time  to  confirm  or  deny  the  informa¬ 
tion  uncertainties  presented  them.”  Gruner  maintains  that  the  rapid  pace  involved  in  these  types  of 
situations  can  exceed  the  capacity  of  the  human  to  comprehend  the  rapid  flow  of  information  pres¬ 
ented  by  complex  systems.  In  the  case  of  the  Vincennes,  the  CIC  team  had  3  minutes  and  40  seconds 
to  make  their  decision.  This  includes  the  time  required  for  the  operators  to  perceive  and  interpret  sen¬ 
sor  data  and  for  the  commanding  officer  to  make  informed  judgments  from  these  data  (Roberts  & 
Dotterway,  1995).  The  result  of  the  human’s  limited  cognitive-processing  capabilities  is  that  the  deci¬ 
sion  makers  may  fail  to  remember  critical  pieces  of  data,  overlook  stored  information,  draw  hasty 
conclusions,  and  produce  flawed  answers. 

Simon  (1978,  p.  273)  states,  “...the  human  information  processing  system... operates  almost  entirely 
serially,  one  process  at  a  time,  rather  than  in  parallel  fashion.  This  seriality  is  reflected  in  the  narrow¬ 
ness  of  its  momentary  focus  of  attention.”  However,  the  AAW  problem  forces  the  decision  maker  to 
operate  in  a  parallel-processing  mode  when  several  contacts  demand  attention  at  the  same  time.  The 
requirement  to  monitor  and  maintain  an  accurate  situation  awareness  for  these  concurrent  contacts, 
over  the  course  of  the  evolving  situation,  imposes  an  additional  load  of  strategically  managing  the 
overall  situation.  Several  researchers  have  argued  that  “managing  the  attentional  and  conceptual  pro¬ 
cesses  that  permit  cogent  SA  involves  significant  cognitive  resources”  (Adams  et  al.,  1995,  p.  91; 
Endsley,  1988).  Having  to  handle  several  concurrent  tasks  can  place  an  unrealistic  cognitive  load  on 
the  decision  maker.  Typical  examples  of  such  tasks  are  prioritizing  contacts  and  the  associated  actions 
to  be  taken  by  the  team,  updating  the  status  of  critical  contacts,  responding  to  the  other  requisite  tasks 
in  the  queue,  and,  more  generally,  of  strategically  managing  the  workload  of  current  multitask  sys¬ 
tems  under  dynamically  changing  scenarios. 

A  major  advantage  offered  by  the  experimental  DSS  is  that  it  should  “buy  time”  for  the  user  by 
(1)  performing  many  of  the  cognitive-processing  tasks  for  the  user  and  (2)  by  presenting  information 
in  graphic  format.  The  DSS  will  synthesize  much  of  the  information  used  to  develop  situation  aware¬ 
ness  and  present  a  coherent  picture  of  the  situation  to  the  user.  This  integrated  picture  will  be  por¬ 
trayed  graphically — ^rather  than  in  the  current  text-based  format — ^which  should  further  reduce  the 
amount  of  time  required  to  assimilate  this  information.  By  performing  several  information-processing 
steps  for  the  decision  maker,  the  decision  maker’s  limited  cognitive  resources  can  be  used  for  the 
types  of  decisions  requiring  human  abilities  (e.g.,  the  decision  on  whether  to  engage). 

WORKING-MEMORY  REQUIREMENTS 

An  essential  information-processing  step  required  by  this  task — and  one  that  levies  a  heavy  load  on 
working  memory — ^involves  integrating  kinematic  and  sensor  variables  and  maintaining  an  awareness 
of  changes  in  these  variables  over  time.  Changes  in  a  contact’s  behavior,  such  as  decreasing  altitude, 
increasing  speed,  changes  in  electronic  emissions,  etc.,  can  provide  key  indicators  of  possible  hostile 
intent.  With  current  systems,  concerning  a  particular  contact,  the  decision  maker  receives  numerous 
reports  from  CIC  team  members  who  provide  various  pieces  of  the  overall  tactical  picture  (such  as 
kinematic  parameter  values,  active  electronic-emitter  identifications,  and  behavioral  responses  of  the 
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contact  in  response  to  queries  by  the  team).  Some  of  this  information  is  also  displayed  in  a  text-based 
format  for  the  user  when  a  contact  is  “hooked”  (that  is,  selected  for  display)  by  the  decision  maker. 
However,  to  recognize  a  change  in  certain  variables,  current  systems  require  the  user  to  retain  param¬ 
eter  values  in  short-term  memory  in  order  to  recognize  a  change  in  the  parameter,  such  as  altitude. 

When  the  decision  maker  is  monitoring  several  concurrent  contacts  (such  as  cycling  through  three 
or  four  contacts  in  a  1 -minute  period),  human  working-memory  capabilities  may  quickly  be  sur¬ 
passed.  To  detect  a  change  in  a  critical  parameter  value,  the  decision  maker  must  maintain  the  param¬ 
eter  values  for  the  contacts  of  interest  in  working  memory  as  he  or  she  cycles  between  several  con¬ 
tacts.  For  example,  the  decision  maker  must  be  able  to  recall  that  contact  7022  was  at  a  14,000-foot 
altitude  1 -minute  ago,  and  then  subtract  the  current  altitude  value  of  10,000  feet,  which  will  then  indi¬ 
cate  the  aircraft  is  in  a  rapid  descent.  The  DSS  was  developed  to  aid  the  decision  maker  by  perform¬ 
ing  several  of  these  cognitive-processing  tasks,  thus,  reducing  the  cognitive  load  for  the  user.  By 
presenting  the  synthesized  picture  of  the  contact’s  behavior  over  time,  through  the  use  of  graphical 
displays,  critical  changes  should  be  immediately  apparent  to  the  user. 

A  second  memory-intensive  task  involves  maintaining  (in  working  memory)  a  current  list  of 
actions  taken  by  team  members,  the  contact’s  response  to  these  actions  taken  by  the  CIC  team,  and 
pending  actions.  Research  has  established  that  “memory  is  limited  and  that  list  maintenance  is  effort¬ 
ful  and  fallible — ^more  so  if  the  list  must  be  ordered,  and  still  more  if  the  membership  of  the  list  must 
be  dynamically  reordered  and  modified  during  retention”  (Bower,  1970,  as  cited  in  Adams  et  al., 
1995,  p.  91).  The  DSS  should  reduce  the  cognitive  effort  required  for  distributing  attention  among 
the  many  contacts  to  be  resolved  and  their  attendant  required  actions.  Working-memory  requirements 
should  be  reduced  by  having  the  DSS  act  as  an  intelligent  “assistant,”  reminding  the  user  what 
actions  are  to  be  taken  and  when. 

A  third  way  the  DSS  will  reduce  memory  and  information-processing  requirements  is  by  display¬ 
ing  templates  depicting  weapons’  envelopes  for  both  the  inbound  contact  and  “own-ship.”  This 
should  facilitate  critical  comparisons  and  judgments  concerning  timing  of  actions.  During  a  scenario, 
decision  makers  have  to  either  rely  on  memory  to  recall  the  launch  range  for  various  weapons  or 
query  a  team  member  for  this  information — ^with  both  of  these  methods  wasting  limited  resources. 
The  high-workload  and  high-tempo  characteristic  of  littoral  scenarios  produce  a  stressful  decision¬ 
making  environment.  The  phenomenon  that  increasing  stress  leads  to  decreasing  working  memory  is 
well  documented  (e.g..  Hockey,  1986).  Moreover,  the  latter  method  for  obtaining  the  desired  informa¬ 
tion  wastes  limited  resources  by  increasing  the  conununications  load  and  requiring  more  time  to  wait 
for  a  team  member’s  response  to  the  query. 

Under  these  high-tempo  and  high-workload  conditions,  human  memory  and  attentional  resources 
can  easily  be  surpassed.  Several  cognitively  resource-intensive  information-processing  steps  are  elim¬ 
inated  for  the  human  decision  maker  by  having  them  performed  by  the  DSS.  We  predict  that  the  deci¬ 
sion  support  tools  will  reduce  the  cognitive  workload  imposed  on  the  decision  maker  in  the  following 
three  ways:  (1)  by  reducing  the  amount  of  information  processing  to  be  performed,  (2)  reducing 
working-memory  requirements,  and  (3)  assisting  the  user  in  allocating  limited  attentional  resources. 

REDUCING  HUMAN  ERROR 

The  study  of  human  cognitive  processes  and  related  error  mechanisms  has  gained  rapidly 
increasing  interest  in  the  past  decade.  Rasmussen  (1987)  argues  that  the  emphasis  in  attempting  to 
understand  human  errors  must  shift  from  tasks  to  the  human-task  mismatch.  For  example,  Gruner 
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(1990,  p.  39),  in  discussing  the  Vincennes  incident,  maintains  that  “the  system  was  poorly  suited  for 
use  by  human  beings  during  rapid  military  action.”  He  ascribes  this  lack  of  suitability  to  a  human- 
machine  mismatch  between  the  rate  of  data  flow  possible  with  modem  computer  systems  that  can 
process  and  display  information  at  phenomenal  data  rates  and  the  “comprehension  capability  of  users 
which  has  remained  almost  static  for  thousands  of  years.”  This  causal  approach  to  understanding 
human  error  is  based  on  the  premise  that  errors  are  rarely  random  and  can  be  traced  to  causes  and 
contributing  factors.  Once  these  contributing  factors  are  identified,  they  can  be  mitigated. 

<  The  impact  and  vulnerability  of  systems  and  human  interfaces,  because  of  incompatibilities 

between  the  way  people  perceive,  think,  and  act,  are  documented  in  popular  and  technical  literature 
(Buck,  1989;  Casey,  1993;  Norman,  1988;  Perrow,  1984;  'Wilson  &  Zarakas,  1978).  Newly  devel¬ 
oped  systems  will  succeed  or  fail  based  on  our  ability  to  minimize  these  incompatibilities  between  the 
characteristics  of  the  things  we  create  and  the  way  we  use  them.  There  are  many  well-documented 
instances  of  critical  systems  or  parameter  changes  going  unnoticed  or  unheeded  because  the  operating 
procedures,  or  the  human  machine  interface,  provided  no  historical  trace.  For  example,  an  unnoticed 
increase  in  altitude  contributed  to  the  shoot  down  of  the  Iranian  airbus  by  a  U.S.  Navy  ship — when 
the  team  mistakenly  believed  the  aircraft  to  be  descending — ^because  there  was  no  historical  trace  to 
make  the  aircraft’s  actual  increasing  altitude  apparent  (Dotterway,  1992).  Five  personnel  in  the 
USS  Vincennes’  combat  information  center,  all  viewing  separate  displays,  reported  the  aircraft  as 
descending,  while  the  Aegis  data  tapes  later  revealed  a  flight  pattern  of  ascent  (Roberts  &  Dotterway, 
1995).  One  of  the  official  investigations  of  this  incident,  the  Fogarty  Report  (1988,  p.  45),  states  that 
“stress,  task  fixation,  and  an  unconscious  distortion  of  data  may  have  played  a  major  role  in  this  inci¬ 
dent.”  A  panel  of  five  psychologists  from  the  American  Psychological  Association  (APA)  who  testi¬ 
fied  before  Congress  concluded  that  there  were  “predictable  failings  of  human  judgment  under 
intense  stress  compounded  by  complex  technology  [which]  clearly  contributed  to  the  accidental 
shooting  of  Iranian  airliner  Flight  655”  (APA,  1988,  p.  4). 

It  is  generally  accepted  that  between  60-80  percent  of  the  accidents  and  malfunctions  in  transporta¬ 
tion,  manufacturing,  process  control,  weapon,  and  other  systems  are  attributable  to  human  error 
(Senders  &  Moray,  1991;  Van  Cott,  1993;  Weiner,  1994).  Reducing  tactical  decision-making  errors  is 
one  goal  of  the  TADMUS  program.  The  following  section  presents  a  brief  review  of  an  experiment 
conducted  to  develop  a  baseline  on  tactical  decision-making  performance  in  response  to  fairly  stress¬ 
ful  scenarios.  The  last  section  of  this  chapter  describes  the  experimental  DSS  modules  and  the  way 
they  are  hypothesized  to  enhance  tactical  decision-making  performance. 
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TADMUS  BASELINE  EXPERIMENT 


Early  research  involved  data  collection  in  the  Decision-Making  Evaluation  Facility  for  Tactical 
Teams  (DEFTT)  Laboratory  using  simulated  existing  shipboard  displays  to  establish  a  baseline  on 
decision-making  performance.  The  purpose  of  this  effort  was  to  document  baseline  decision-making 
performance  for  experienced  naval  officers.  During  the  baseline  phase  of  testing,  a  detailed  under¬ 
standing  was  developed  of  the  cognitive  processes  underlying  the  various  tasks  involved  in  situation 
assessment — and  where  the  bottlenecks  occur.  This  understanding  was  then  used  to  design  the  way 
the  information  is  presented  to  the  user  to  facilitate  performance  of  the  required  tasks. 

SUBJECTS 

This  study  focused  on  the  command-level  decision  makers  of  an  antiair-warfare  team  on  an  Aegis 
cruiser — ^the  commanding  officer  and  the  tactical  action  officer.  Subjects  in  the  study  consisted  of 
6  commanding  officer/tactical  action  officer  teams  drawn  from  12  active-duty  Naval  personnel;  some 
were  from  training  commands,  while  others  were  from  operational  commands  aboard  ship  or 
assigned  to  group  staffs. 

PROCEDURE 

Data  were  collected  in  the  DEFTT  Laboratory,  a  six-station  test-bed  environment  that  simulates 
console  positions  in  a  Navy  Aegis  cruiser  combat  information  center.  (For  a  detailed  description  of 
the  DEFTT  Laboratory,  see  Hutchins,  1996.)  Four  stations  were  filled  by  confederates  (active-duty 
Navy  personnel)  who  play  antiair-warfare  support-team  member  roles.  These  roles  included  the 
antiair-warfare  coordinator,  identification  supervisor,  tactical  information  coordinator,  and  electronic 
warfare  supervisor.  After  approximately  Ij  hours  of  orientation  concerning  the  laboratory  and  train¬ 
ing  using  the  computer  consoles,  the  subjects  engaged  in  four  scenarios.  Each  scenario  was  about 
25  minutes  long  and  contained  between  11  and  14  contacts  of  interest  per  scenario,  in  addition  to 
numerous  background  contacts. 

TREATMENT  OF  DATA 

Team  communications  were  recorded  on  a  multichannel  audio  recorder;  these  communications 
included  all  intrateam  exchanges,  as  well  as  all  communications  with  simulated  off-ship  personnel. 
Audio  tapes  were  used  to  produce  verbatim,  time-stamped  transcripts  of  all  team  communications.  A 
modified  version  of  the  TapRoot®  Incident  Investigation  System  (Paradies,  1991;  Paradies  &  Unger, 
1991)  was  then  applied  to  identify  errors.  The  objective  was  to  identify  tactically  significant  errors 
committed  during  the  scenario.  Tactically  significant  errors  were  defined  as  those  errors  that  may  lead 
to  loss  of  life  or  significant  political  embarrassment.  The  following  criteria  were  used  for  counting  an 
error  as  tactically  significant:  (1)  loss  of  situation  awareness,  (2)  failure  to  take  defensive  action 
when  within  the  weapon’s  range  of  an  approaching  contact,  or  (3)  a  violation  of  rules  of  engagement 
(ROE).  Video  recordings  were  made  of  the  commanding  officer  and  tactical  action  officer  computer 
screens.  All  audio  and  video  recordings  were  then  analyzed  in  detail.  (For  a  more  detailed  coverage 
of  the  methodology  and  results,  see  Hutchins  and  Westra,  in  preparation.) 

RESULTS 

The  complex,  time-constrained,  decision-making  situations  embodied  in  the  experimental  scenarios 
resulted  in  a  large  number  of  decision  errors.  The  mean  number  of  tactically  significant  errors  docu¬ 
mented  across  6  teams  and  4  scenarios  was  14;  the  number  of  errors  ranged  from  9  to  22.  The 
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standard  deviation  was  3.7.  Subjects  performed  an  average  of  50  percent  of  the  required  behaviors  as 
specified  in  the  ROE.  The  ordinal  agreement  between  three  raters  (navy  subject  matter  experts)  on 
error  count  ranks  firom  TapRoot®  analyses  was  computed.  Results  showed  a  high  degree  of  agree¬ 
ment  with  the  Kendall’s  W  of  0.93,  indicating  that  93  percent  of  the  possible  rank  variance  is 
accounted  for. 

Decision-Making  Errors 

Detailed  examinations  of  the  information-processing  sequences  performed  during  tactical  decision¬ 
making  have  revealed  a  variety  of  errors.  On  average,  subjects  failed  to  take  required  actions  about 
half  of  the  time.  Explanations  based  in  the  cognitive  psychology  literature  have  been  pursued,  since  a 
major  goal  of  the  TADMUS  program  is  to  develop  a  DSS  based  on  an  understanding  of  the  way 
human  decision  makers  actually  process  information  under  rapidly  evolving  situations. 

The  majority  of  documented  errors  involved  errors  of  omission,  that  is,  “failure  to  take  defensive 
measures”  and  “failure  to  adhere  to  ROE.”  Failure  to  take  defensive  measures  included  failure  to  take 
actions  to  defend  own-ship  when  an  approaching  aircraft  had  reached  its  weapon’s  release  range.  An 
example  involved  a  case  where  two  contacts  were  within  the  specified  ROE  limit,  yet  no  action  had 
been  taken.  The  types  of  actions  included  in  the  “failure  to  adhere  to  ROE”  category  include  failure  to 
take  action  concerning  the  following  listed  and  defined  items: 

a.  Issuing  warnings  is  part  of  the  usual  identification  process  and  involves  three  levels  of 
warnings  with  increasing  levels  of  urgency. 

b.  Establish  friendly-force  criteria  refers  to  establishing  a  plan  with  other  friendly  ships  in 
the  area  to  coordinate  how  they  will  respond  to  potential  threats. 

c.  Changes  in  kinematics/identification  friend  or  foe — ^subjects  are  expected  to  notice  sig¬ 
nificant  kinematic  changes  and/or  identification  friend  or  foe  parameter  changes. 

d.  Other  identification  procedures  includes  actions  such  as  illuminating  with  fire-control 
radar. 

The  other  major  category  of  tactically  significant  error  involved  “loss  of  SA.”  These  loss  of  SA 
errors  were  grouped  under  errors  of  commission  and  errors  of  omission  and  then  further  categorized 
into  subgroups.  Fifty-five  percent  of  the  loss  of  SA  errors  involved  taking  the  wrong  action  (error  of 
commission),  while  45  percent  of  the  errors  involved  failing  to  take  some  required  action  (error  of 
omission).  Error  categories  included  under  errors  of  commission  involved  incorrectly  engaging  a 
track  (3  percent),  incorrectly  warning  a  track  (29  percent),  other  incorrect  actions  (16  percent),  and 
incorrect  reporting  (7  percent).  The  two  instances  of  incorrectly  engaging  an  aircraft,  which  were  F-1 
Mirage  aircraft,  were  considered  errors  because  the  decision  maker  failed  to  take  certain  actions  prior 
to  engaging — not  necessarily  because  the  aircraft  should  not  have  been  engaged.  The  actions  that  the 
decision  makers  failed  to  take  involved  ascertaining  the  identification  of  the  aircraft  for  one  case  and 
failure  to  warn  and  illuminate  the  aircraft  prior  to  engaging  for  the  second  case.  Most  instances  of 
incorrectly  issuing  warnings  to  the  aircraft  involved  issuing  the  warning  when  the  aircraft  was  within 
its  territorial  airspace  (that  is,  inside  the  12-nautical-mile  limit  which  is  internationally  recognized  as 
under  control  of  that  nation)  or  issuing  a  warning  at  a  level  different  from  what  was  required.  Other 
incorrect  actions  included  illuminating  the  aircraft,  “locking  up”  with  radar,  or  ordering  the  aircraft  to 
divert  when  the  aircraft  was  still  within  its  territorial  airspace.  Incorrect  reporting  involved  inaccurate 
reports  on  the  stams  of  the  tactical  situation  (such  as  indicating  to  the  battle-group  commander  that 
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certain  actions  had  been  taken  when  they  had  not,  misidentification  of  an  aircraft,  or  omitting  critical 
tracks  from  a  report). 

Errors  of  omission  categorized  under  the  “loss  of  SA”  category  included  (a)  failure  to  identify  or 
attend  to  a  contact;  (b)  failure  to  take  action  (e.g.,  to  issue  “hold-fire”  when  a  contact  turned  out¬ 
bound);  (c)  failure  to  recognize  a  threat  (e.g.,  designating  an  aircraft  as  a  nonthreat  because  it  had 
passed  its  closest  point-of-approach,  yet  it  was  still  within  missile-launch  range);  (d)  instances  of  con¬ 
fusion  or  forgetting  (e.g.,  forgetting  or  ignoring  critical  data,  forgetting  whether  or  not  the  aircraft  had 
been  warned,  illuminated,  or  “locked-on,”  or  forgetting  the  aircraft’s  response,  or  lack  of  response  to 
these  actions,  forgetting  the  status  of  a  contact,  and  confusing  contacts);  (e)  misperception  of  data 
(e.g.,  reporting  a  contact  as  turning  outbound  when  it  is  still  inbound);  (f)  unclear  communication 
(issuing  vague  orders  regarding  actions  to  be  taken  by  a  team  member,  such  as  failure  to  specify 
which  weapon  system  is  to  be  used  or  which  contact  is  to  be  engaged). 

Cognitive  Explanations 

The  cause  of  failures  to  take  required  actions  is,  in  many  cases,  attributed  to  the  extremely  high 
task  demands  levied  on  the  decision  maker  by  the  scenario  and  the  human  decision  maker’s  limited 
attentional  resources.  Many  cases  are  also  attributed  to  working-memory  limitations.  Maintaining  an 
awareness  of  the  status  of  each  contact  and  the  status  of  many  actions  to  be  taken  by  the  antiair- 

warfare  team— which  actions  have  been  taken  and  what  the  contact’s  response  to  the  action  was _ 

severely  taxes  the  decision  maker’s  working  memory.  The  high  workload  entailed  in  the  scenarios 
produces  a  highly  time-compressed  decision-making  situation.  This  time-compressed  decision¬ 
making  situation — where  attentional  resources  and  working-memory  capacity  are  limited — do  not 
allow  the  decision  maker  to  maintain  accurate  SA  for  all  tracks  at  any  given  time.  We  anticipate  that 
the  decision  support  modules  in  the  DSS  will  mitigate  these  types  of  errors. 

Human  information-processing  capabilities  are  not  well  suited  to  dealing  with  a  “multiplicity  of 
simultaneous  and  disjointed  tasks.  Thoughtful  attention  is  modular:  People  can  consciously  think 
about  only  one  thing  at  a  time”  (Adams  et  al.,  1995,  p.  92).  As  a  result,  they  do  not  handle  interrup¬ 
tions  very  well.  Research  indicates  that  when  an  operator  is  faced  with  as  few  as  two  tasks  consisting 
of  merely  detecting  or  recognizing  simple  signals,  a  cost  may  be  incurred  in  terms  of  a  significant 
loss  in  sensitivity  or  time  that  can  be  allocated  to  either  by  the  requirement  to  divide  or  switch  atten¬ 
tion  between  them  (Broadbent,  1957;  Schneider  &  Detweiler,  1988;  Swets,  1984). 

The  memory  demands  of  managing  complex,  multitask  situations  can  easily  surpass  human  limita¬ 
tions.  The  decision  maker  must  not  forget  any  of  the  contacts  or  tasks  requiring  action.  In  addition  to 
remembering  all  the  tasks  needing  attention,  however,  are  the  complexities  entailed  in  keeping  track 
of  the  data  and  substeps  associated  with  each  contact  and  prior  action.  The  aviation  literature  provides 
many  examples  of  incidents  with  explanations  similar  to  the  root  causes  for  errors  that  were  found  in 
the  TADMUS  program.  One  category  includes  the  potentially  disastrous  effects  of  interruptions  in  the 
task  for  air  traffic  controllers  and  pilots.  Similarly,  in  the  AAW  environment,  momentary  intervening 
attention  to  another  task  or  contact,  or  an  interruption  in  a  procedure,  can  leave  the  procedure  or  pro¬ 
cessing  of  a  contact  incomplete  with  potentially  catastrophic  results. 

A  fairly  consistent  pattern  of  tactical  decision-making  errors  was  documented  from  data  collected 
during  the  baseline  data-collection  period.  The  root  causes  of  these  errors  were  traced  to  cognitive 
mechanisms  such  as  limited  attentional  resources  and  working-memory  limitations.  By  developing  an 
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understanding  of  the  pattern  and  types  of  errors  most  frequently  observed  in  this  task  domain,  we 
hope  to  provide  a  DSS  that  will  mitigate  these  errors. 
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DECISION  SUPPORT  SYSTEM  DESCRIPTION 


A  DSS  was  designed  for  naval  shipboard  command-level  decision-makers  to  enhance  decision 
making  under  the  ambiguous,  high-workload  and  high-tempo  conditions  characteristic  of  near-land 
operations.  Design  of  the  prototype  DSS  was  based  on  (1)  an  understanding  of  the  cognitive  strate¬ 
gies  people  bring  to  bear  when  dealing  with  the  types  of  decisions  required  in  tactical  decision-mak¬ 
ing  and  (2)  applying  human-system  interface  design  principles  which  are  expected  to  help  compen¬ 
sate  for  human  cognitive-processing  limitations. 

Major  innovations  are  that  the  new  DSS  (a)  helps  the  user  anticipate  future  events  by  portraying  the 
contact’s  history  (not  simply  contact  positions  at  separate  time  slices),  (b)  helps  identify  stored  pat¬ 
terns  of  events,  and  (c)  shows  planned  responses  for  anticipated  events,  thereby  gaining  time  for  the 
decision-maker  (Tolcott,  1993).  A  corollary  benefit  is  that  the  underlying  design  principles  incorpo¬ 
rated  in  the  DSS  provide  a  better  match  between  those  tasks  performed  by  the  computer  and  those 
performed  by  the  human.  For  example,  research  indicates  people  are  better  at  selecting  and  coding 
information  than  they  are  at  integrating  it  (Dawes,  1971).  The  DSS  is  a  data-fusion  engine  that  pres¬ 
ents  all  known  information  in  a  highly  synthesized  format. 

Currently  available  decision  support  systems  typically  provide  indications  of  the  presence  of  vari¬ 
ous  data  items  and  their  values.  Indications  of  the  relations  between  features,  structural  similarity 
between  sequences  of  features,  or  evidence  of  underlying  relationships  among  features  is  usually  not 
depicted.  The  TADMUS  decision-support  toots  were  designed  to  provide  precisely  these  types  of  evi¬ 
dence.  All  of  the  DSS  modules  focus  on  providing  information  to  the  user  to  support  situation  aware¬ 
ness.  These  modules  will  be  discussed  in  terms  of  the  underlying  theoretical  concepts  and  in  terms  of 
the  expected  payoff  for  each  decision-support  tool. 

The  experimental  DSS  receives  tactical-data  input  from  many  sources,  integrates  the  data  to  present 
a  coherent  picture,  and  displays  it  in  various  ways  for  aiding  various  subtasks.  (These  tactical  data  are 
inserted  directly  into  the  DSS  from  the  scenario  generator  in  the  DEFTT  Laboratory.  This  information 
would  come  from  the  sensor  systems  found  aboard  ship  in  the  operational  application  of  DSS.) 

DECISION  SUPPORT  SYSTEM  MODULES 

The  DSS  was  developed  for  the  command-level  decision  makers — ^the  commanding  officer  and  the 
tactical  action  officer — within  a  Navy  combat  information  center.  These  two  officers  are  responsible 
for  the  following  combat-related  decisions:  (1)  recognizing  and  correctly  interpreting  the  nature  of  the 
threat,  (2)  predicting  which  preplanned  responses  and/or  countermeasures  may  be  effective  if  the  situ¬ 
ation  evolves  in  certain  ways,  and  (3)  interpreting  information  to  determine  whether  the  situation  is 
actually  evolving  in  a  particular  way.  An  additional  task  is  to  ensure  that  team  members  provide  criti¬ 
cal  information  that  would  enable  the  team  to  resolve  ambiguities.  The  six  modules  making  up  the 
DSS  were  designed  to  facilitate  the  decision-maker’s  performance  of  the  tasks  listed  above  and,  in 
general,  to  promote  an  accurate  situation  awareness  while  minimizing  the  decision  maker’s  cognitive- 
processing  requirements. 

The  layout  for  the  prototype  DSS  display  modules  is  shown  in  figure  1.  These  modules  are 
arranged  in  an  increasing  level  of  information  complexity  from  the  top  of  the  display  to  the  bottom. 
The  top  half  of  the  display  contains  the  track  summary,  track  history,  and  the  response  manager  mod¬ 
ules.  (The  track  history  module  occupies  the  rectangle  in  the  upper-right  comer  and  in  the  circular 
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Figure  1 .  Decision  Support  System  Modules. 

display  below  track  summary  in  the  upper  left.  These  two  areas  present  different  views  of  the  same 
information  and  are  thus  considered  as  one  module.)  These  three  modules  focus  on  analyzing  and 
identifying  a  single  contact  and  would  be  used  to  make  a  quick  shoot/no-shoot  decision.  The  lower 
half  of  the  display  presents  the  basis  for  assessment,  comparison  to  normal  values,  and  the  track 
priority  list  and  alerts  modules.  Basis  for  assessment  and  the  comparison  to  normal  values  modules 
provide  more  detailed  analysis  of  a  single  contact,  while  the  track  priority  list  and  alerts  module  (bot¬ 
tom  of  the  display)  presents  information  on  all  contacts  currently  requiring  attention  or  action.  The 
actual  prototype  DSS  comprises  the  six  display  modules  shown  in  figure  2. 

Track  Summary 

Track  summary  presents  the  same  contact  parametric  information  current  systems  provide,  but  with 
an  improved  human-computer  interface  format  and  the  addition  of  a  DSS  threat-assessment  capabil¬ 
ity.  Here,  all  data  items  are  appropriately  labeled  and  aligned  to  aid  the  user  in  extracting  key  parame¬ 
ters. 

Track  History 

When  a  rapid  decision  is  required  to  determine  whether  the  contact  should  be  engaged,  the  track 
history  and  response  manager  modules  should  present  useful  assistance  in  two  ways:  by  facilitating 
the  decision  maker’s  use  of  a  recognitional  strategy  when  developing  a  threat  assessment  (via  track 
history)  and  by  assisting  the  decision  maker  in  performing  the  requisite  actions.  For  tasks  involving 
rapid  decision-making  (such  as  several  seconds  to  1  or  2  minutes),  a  recognitional  strategy  appears  to 
be  highly  efficient  (Klein,  1993).  The  track  history  module  was  designed  to  support  the  recognition- 
primed  decision  model  of  decision-making.  To  generate  a  reasonable  course  of  action,  the  decision 
maker  must  accurately  identify  familiar  elements  in  the  situation.  The  objective  for  this  module  is  to 
facilitate  the  contact  identification  process  by  providing  information  integrated  to  support  a  recogni¬ 
tional  decision  strategy.  The  track  history  module  presents  a  highly  synthesized  view  of  the  situation 
concerning  a  specific  contact. 

The  track  history  module  depicts  a  contact’s  speed,  altitude,  course,  and  range  on  a  two-dimen¬ 
sional  graphical  display,  along  with  a  geometric  representation  of  both  the  contact’s  weapon-release 
envelope  and  own-ship’s  weapons  coverage.  A  large  amount  of  parametric  data  is  portrayed  graphi¬ 
cally  for  rapid  assimilation  by  the  user.  The  track  history  module  is  designed  to  be  used  when  the 
decision-maker  has  to  make  a  rapid  shoot/no-shoot  decision.  Changes  in  the  contact’s  speed,  course. 
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Figure  2.  The  Decision  Support  System  Display. 


altitude,  or  range  are  immediately  apparent  with  this  graphical  depiction  of  the  track  history.  This 
graphical  representation  was  hypothesized  to  be  particularly  useful,  because  in  previous  systems  the 
user  had  to  remember  the  previous  parameter  values  (e.g.,  altitude)  and  compare  it  with  the  current 
values  for  those  parameters.  However,  as  short-term  memory  degrades  under  stress,  the  user  may  not 
be  able  to  accurately  perform  this  function. 

There  are  many  well-documented  instances  of  critical  systems  or  parameter  changes  going  unno¬ 
ticed  or  unheeded,  because  the  operating  procedures,  or  the  human-machine  interface,  provided  no 
historical  trace.  For  example,  an  unnoticed  increase  in  altitude  contributed  to  the  shoot  down  of  the 
Iranian  airbus — when  the  team  mistakenly  believed  the  aircraft  to  be  descending — ^because  there  was 
no  historical  trace  to  make  the  aircraft’s  actual  increasing  altitude  apparent  (Dotterway,  1992). 

Response  Manager 

The  response  manager  provides  assistance  in  using  preplanned  responses.  This  decision  support 
module  was  designed  specifically  to  mitigate  errors  documented  in  previous  research  (Hutchins  & 
Kowalski,  1993;  Hutchins  &  Westra,  1995).  The  response  manager  was  developed  to  (1)  provide  sup¬ 
port  to  the  user  regarding  required  actions  and  when  they  need  to  be  taken  and  (2)  to  lessen  the  task 
load  imposed  on  the  user’s  limited  attentional  resources  during  high-contact  density.  The  following 
tasks  severely  tax  the  decision-maker’s  working  memory:  maintaining  an  awareness  of  the  status  of 
each  contact  and  the  status  of  many  actions  to  be  taken  by  the  CIC  team  (e.g.,  issuing  warnings,  illu¬ 
minating  with  radar,  executing  electronic-support  measures  packages,  verifying  airspace,  readying 
self-defense  systems,  making  reports  to  the  battle-group  commander,  etc.) — as  well  as  remembering 
the  contact’s  response  to  these  various  actions. 

During  TADMUS  baseline  runs,  the  majority  of  documented  errors  involved  errors  of  omission. 
These  included  the  failure  to  take  defensive  measures,  failure  to  adhere  to  ROE,  as  well  as  about  half 
of  the  loss  of  SA  errors.  (See  Hutchins  &  Westra,  1995,  for  a  detailed  coverage  of  the  documented 
errors.)  The  cause  of  these  failures  to  take  required  actions  is,  in  many  cases,  attributed  to  the  very 
high-task  demands  levied  on  the  decision  maker  by  the  scenario.  The  scenarios  were  intentionally 
developed  to  be  highly  stressful  (albeit  realistic)  by  including  high  levels  of  ambiguity,  workload,  and 
time-pressure.  The  phenomenon  that  increasing  stress  leads  to  decreasing  working  memory  is  well 
documented  (e.g..  Hockey,  1986).  Effective  maintenance  of  the  queue  of  pending  tasks  requires  con¬ 
siderable  cognitive  effort. 

Competent  management  of  the  task  constellation  requires  that  the  decision  maker  respond  to  each 
task,  ideally,  at  the  most  efficient  possible  moment  or,  minimally,  before  it  is  too  late.  This  requires 
the  person  to  stay  abreast  of  the  urgencies,  opportunities,  and  constraints  on  all  of  the  tasks.  The 
response  manager  module  graphically  depicts  preplanned  responses  that  need  to  be  taken  for  a 
selected  contact,  using  a  series  of  bars  (on  a  range  scale)  that  show  the  earliest  and  latest  time  each 
action  can  be  taken  to  be  effective.  This  module  also  depicts  the  contact’s  current  speed  and  range  via 
a  moving  pointer  that  indicates  where  the  contact  is  in  relation  to  the  various  actions  which  need  to  be 
taken.  The  response  manager  module  thus  cues  the  decision  maker  to  take  actions  specified  by  the 
ROE  or  ship’s  battle  orders.  Color  coding  is  used  to  keep  track  of  which  actions  have  been  taken  and 
which  actions  remain  to  be  taken. 

Basis  for  Assessment 

Basis  for  assessment  is  based  on  a  model  of  a  cognitive  strategy  employed  in  making  decisions 
where  the  decision  maker  is  confronted  with  a  situation  involving  contradictory  or  incomplete 


24 


information.  In  this  situation,  the  decision  maker  constructs  a  causal  model  that  explains  the  available 
evidence.  This  strategy  is  known  as  explanation-based  reasoning,  or  story  generation  (Pennington  & 
Hastie,  1993).  In  this  approach,  available  data  are  assembled  into  explanatory  structures,  with  one 
structure  for  each  possible  conclusion.  Each  of  the  explanations  attempts  to  explain  how  every  piece 
of  data  can  be  accounted  for  in  support  of  each  conclusion,  even  though  some  of  the  data  items  would 
naturally  contradict  reaching  some  conclusions.  Contradictory  data  are  explained  through  the  use  of 
internal  assumptions.  A  fixed  number  of  predefined  possible  conclusions  are  assumed,  and  each  data 
item  points  directly  to  one  of  those  possible  conclusions. 

Avoiding  Decision  Bisses.  Basis  for  threat  assessment  provides  a  contact  threat  assessment  and 
presents  all  the  information  used  to  form  the  assessment.  Threat  assessment  will  usually  generate 
multiple  hypotheses  to  explain  the  available  evidence.  This  occurs  because  many  of  the  contacts 
behave  in  a  way  that  greatly  complicates  determining  whether  they  are  a  threat  or  nonthreat  due  to  the 
inherent  ambiguity  in  the  scenarios.  In  addition  to  reflecting  a  naturalistic  decision  strategy  based  on 
explanation-based  reasoning,  this  decision  support  module  should  have  the  corollary  benefit  of  reduc¬ 
ing  decision  bias.  Human  decision-makers  have  been  shown  to  be  deficient  in  generating  alternatives. 
Specifically,  in  simation  assessment  they  tend  to  generate  only  a  few  hypotheses  based  on  early  data 
and  find  it  difficult  to  enlarge  their  hypothesis  set,  even  in  the  face  of  contradictory  data  (Tolcott, 
1991).  Threat  assessment  presents  all  the  supporting  evidence,  counter-evidence,  and  assumptions  the 
decision  maker  would  need  to  accept  the  presented  hypothesis  (that  is,  threat  or  nonthreat).  The 
advantage  is  that  the  decision  maker  should  be  less  susceptible  to  many  of  the  typical  biases  which 
are  well  documented  in  the  decision-making  literature,  such  as  “availability”  bias,  “confirmation” 
bias,  effects  of  “framing,”  etc.  (Kahneman,  Slovic,  &  Tversky,  1982). 

By  presenting  all  available  evidence  in  a  structured  way,  grouped  under  the  three  evidence  catego¬ 
ries  mentioned  above,  the  decision  maker  is  less  likely  to  make  a  decision  based  on  a  subset  of  the 
available  evidence.  The  decision-maker  will  also  be  made  explicitly  aware  of  the  absence  of  data  that 
may  be  as  important  as  the  presence  of  data.  For  example,  when  studying  Army  intelligence  analysts, 
it  was  found  that  the  confirmation  bias  could  be  reduced  by  displays  that  made  explicit  the  uncertain¬ 
ties  about  enemy-unit  locations  (Tolcott  &  Marvin,  1988).  In  research  conducted  to  investigate 
whether  Navy  decision-makers  responding  to  antiair-warfare  scenarios  would  be  susceptible  to  these 
biases,  the  results  supported  earlier  research  findings.  A  study  was  designed  to  assess  whether  naval 
personnel,  trained  and  experienced  in  antiair  operations,  exhibit  biases  when  performing  their  normal 
duties.  Results  strongly  supported  existence  of  the  availability,  representativeness,  contrast,  and  con¬ 
firmation  biases  in  the  surface  antiair  warfare  context  (Barnett,  Perrin,  &  Walrath,  1993).  Basis  for 
assessment  should  help  ameliorate  the  effects  of  these  pervasive  biases. 

Avoiding  “Blue-On  Blue”  Engagements.  A  second  advantage  of  the  basis  for  assessment  mod¬ 
ule  is  that  it  should  reduce  the  occurrence  of  “blue-on-blue”  incidents  (fratricide)  where  a  decision 
maker  mistakenly  identifies  a  friendly  contact  as  an  enemy.  The  decision  maker  may  be  better  able  to 
weigh  all  the  information  about  a  contact  and  reach  a  more  accurate  assessment  regarding  its  threat 
potential  when  the  DSS  does  the  following:  points  out  the  counter-evidence  associated  with  a  particu¬ 
lar  threat  assessment  category  and  lists  the  assumptions  one  has  to  “buy  into”  to  accept  the  conclusion 
that  the  contact  is  a  threat. 

Comparison  to  Normal  Values 

The  comparison  to  normal  values  module  compares  known  information  about  a  contact  with 
information  representative  of  specific  types  of  contacts.  This  module  was  designed  to  support  the  user 
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in  performing  feature  matching.  Klein  and  associates  (Klein,  1993)  found  feamre  matching  to  be  the 
predominant  cognitive  strategy  used  by  decision  makers  when  performing  situation  assessment.  The 
decision  maker  matches  the  features  of  the  present  situation  with  a  template,  or  mental  model,  of  a 
previous  situation.  Comparison  to  normal  values  presents  a  comparison  of  data  associated  with  the 
selected  contact  with  exemplars  for  other  types  of  contacts  (i.e.,  threat  or  nonthreat)  and  graphically 
depicts  whether  the  selected  contact  is  a  fit  or  misfit  with  these  categories.  Discrete  coding  of  key 
variables  is  used  to  determine  whether  a  contact’s  data  fall  within  the  specific  ranges  that  categorize 
threat  versus  nonthreat. 

Track  Priority  List  and  Alerts 

The  track  priority  list  and  alerts  module  focuses  on  several  contacts  at  once  and  allows  the  user  to 
monitor  more  than  one  contact.  This  module  presents  multiple-contact  information,  focusing  on  high- 
priority  contacts,  the  next  action  to  be  taken  and  status  of  that  action  (e.g.,  immediate,  watch,  low 
priority),  the  last  alert  given  concerning  the  contact  and  the  time  it  was  given,  and  an  alert-history 
function.  The  user  can  click  on  the  last  alert  to  view  a  history  window  that  displays  all  previous  alerts 
received  regarding  this  contact.  Alerts  are  based  on  preset  criteria  for  key  events  and  required 
responses.  Lines  included  in  this  decision  support  module  are  ordered  by  the  operational  priority 
assigned  to  the  corresponding  contact. 

One  type  of  tactically  significant  error  observed  during  early  TADMUS  experiments  was  a  failure 
to  attend  to  a  contact  of  interest.  This  is  attributed  to  the  high  workload  imposed  by  the  ambiguous 
contacts  and  the  very  limited  time  decision  makers  have  to  process  contacts.  The  track  priority  list 
and  alerts  module  was  designed  to  ensure  that  the  user  is  made  aware  of  new  contacts  of  interest  and 
their  status.  This  module  will  prompt  the  user  on  required  actions  and  specify  when  these  actions  are 
to  be  taken.  The  behavior  that  triggered  the  last  alert  is  also  presented  along  with  the  capability  to 
review  the  history  of  alerts  for  any  selected  contact. 

DECISION  SUPPORT  SYSTEM  FEATURES 

In  summary,  key  features  of  the  prototype  DSS  include  the  ability  to: 

•  Track  multiple  hypotheses 

•  Present  patterns  of  tactical  activities 

•  Develop  explanations  for  observed  events 

•  Evaluate  the  plausibility  of  explanations 

•  Use  graphics  to  support  intuitive  processes  and  reduce  cognitive-processing  requirements 

•  Prompt  the  user  for  appropriate  responses 

•  Support  the  decision  maker’s  cognitive  process 
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DISCUSSION 


Failure  to  take  appropriate  actions  may  be  explained  by  the  limited  resource  capacity  of  human 
memory.  In  these  scenarios,  a  large  number  of  contacts  are  monitored  for  changes  in  any  of  several 
key  parameters.  Three  modules  in  the  DSS  are  hypothesized  to  assist  with  recognizing  a  problem  and 
taking  the  appropriate  actions:  the  track  history,  the  response  manager,  and  the  track  priority  list  and 
alert  modules. 

Features  offered  by  the  DSS  to  address  errors  attributed  to  limited  attentional  resources  include 
focusing  attention  on  (1)  high-priority  contacts  (i.e.,  track  priority  list  and  alerts),  as  well  as  on 
(2)  missing  data  (e.g.,  basis  for  assessment),  and  (3)  enabling  the  decision  maker  to  use  more  data 
than  are  typically  used  in  current  systems  (e.g.,  track  history,  comparison  to  normal  values).  Current 
systems  require  the  user  to  retain  previous  contact  data  in  memory  to  compare  with  current  values  for 
critical  parameters.  Current  systems  also  require  the  user  to  rely  on  recalling  vast  amounts  of 
information  from  training  and  experience.  Presenting  all  known  data  on  a  contact  in  a  synthesized 
way  should  reduce  working-memory  requirements  and  facilitate  recognition.  Additional  potential  per¬ 
formance-enhancement  features,  offered  by  the  DSS,  include  displaying  the  complete  kinematic  con¬ 
tact  history,  presenting  graphic  displays  of  location  and  trends,  highlighting  missing  data,  providing 
alerts,  and  providing  assessments  of  current  contact  identity  that  go  beyond  what  existing  systems 
currently  present. 

Focusing  the  user’s  attention  on  trend  and  history  data  should  decrease  the  cognitive  workload 
imposed  by  these  scenarios  where  many  contacts  must  be  identified  and  responded  to  under  severe 
time  constraints.  Similarly,  delineating  trend  and  history  data  can  assist  in  identifying  a  contact  where 
noticing  changes  in  critical  parameters  is  essential.  Presentation  of  trend  and  history  data,  as  well  as 
basis  for  assessment  and  comparison  to  normal  values,  should  also  mitigate  cognitive  “tunnel-vision” 
effects,  where  the  decision  maker  attends  to  a  smaller  number  of  cues  when  under  stress. 

The  notion  of  time  is  an  important  characteristic  of  situation  awareness  (Harwood,  Barnett,  & 
Wickens,  1988).  The  past  is  critical  to  understanding  the  present,  and  both  past  and  present  informa¬ 
tion  must  be  used  to  predict  future  events  (Shrestha  et  al.,  1995).  Endsley  (1988)  referred  to  the  “pro¬ 
jection  of  their  (perceived  elements)  status  in  the  near  future”  when  discussing  situational  awareness. 
However,  Endsley  also  noted  the  task  of  attending  to  incoming  information  and  subsequently  predict¬ 
ing  future  events  places  a  heavy  load  on  working  memory.  Several  decision  support  modules  were 
developed  to  assist  the  user  in  remaining  aware  of  the  contact’s  history  and  changes  over  time. 
Remembering  which  actions  are  to  be  taken  at  what  time  levies  an  additional  burden  by  placing  a 
heavy  load  on  working  memory.  A  secondary  time  savings  should  be  achieved  by  the  DSS  acting  as 
an  intelligent  “advisor,”  that  is,  by  assisting  the  decision  maker  in  knowing  what  actions  to  take, 
when  to  take  them,  and  which  actions  have  already  been  taken.  A  tertiary  time  savings  can  be 
achieved  by  including  a  template  depicting  the  weapons’  release  ranges  so  the  decision  maker  does 
not  need  to  rely  on  fallible  human  memory  or  querying  a  team  member  about  weapons  ranges. 
Information-processing  time  can  be  saved  for  the  decision  maker  by  having  the  DSS  graphically 
depict  a  synthesized  view  of  the  contact’s  behavior  over  time,  along  with  the  contact’s  weapons  enve¬ 
lope  in  relation  to  both  own-ship’s  radar  and  weapon  coverage. 
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CONCLUSIONS 


The  research  reported  here  focuses  on  developing  a  DSS  that  reflects  the  natural  decision-making 
strategies  of  humans.  Presenting  synthesized  information  in  the  form  of  graphic  presentations  is 
expected  to  reduce  the  cognitive-processing  load  for  the  decision  maker  when  performing  situation 
assessment.  The  intention  is  to  aid  the  decision  maker  by  providing  information  in  a  way  that  will 
minimize  the  need  to  maintain  information  in  working  memory,  reduce  information-processing 
demands,  help  focus  attentional  resources  on  the  highest  priority  contacts,  remind  the  user  to  take 
needed  actions,  help  make  decisions  under  stress,  and  support  higher  levels  of  situation  awareness. 

Decision  support  systems  require  that  the  human’s  strengths  be  used  in  synergy  with  the  advan¬ 
tages  offered  by  the  DSS.  Limitations  associated  with  the  current  generation  of  automated  decision 
aids  include  the  idea  that  (1)  they  cannot  adequately  capture  the  expertise  developed  by  experience 
over  time  and  (2)  since  all  contingencies  cannot  be  anticipated,  the  expert’s  abilities  to  use  intuition  is 
indispensable  (Mosier,  1996).  Mosier’s  review  of  the  limitations  of  automated  decision  systems 
delineates  the  characteristics  of  human  expertise  that  surpass  the  capabilities  of  automated  systems. 
These  include  the  human  capacity  for  creativity,  adaptability,  the  ability  to  incorporate  experience,  the 
presence  of  a  broad  focus,  analogical  reasoning,  and  common-sense  knowledge.  The  goal  for  the 
DSS  is  to  capitalize  on  the  strengths  of  the  human  along  with  the  advantages  provided  by  the  decision 
support  system. 

TESTING  THE  DSS 

The  prototype  DSS  display  modules  are  currently  being  empirically  evaluated  in  the  simulated 
tactical  environment  provided  in  the  DEFTT  Laboratory.  Experienced  naval  decision  makers  engage 
in  experimental  scenarios  with  and  without  access  to  the  DSS  display.  The  various  decision  support 
modules  will  be  tested  individually  and  in  combination  in  future  experiments.  The  following  informa¬ 
tion  will  be  collected:  data  on  reduction  of  errors,  improvements  in  users’  situation  awareness  scores, 
changes  in  communication  patterns,  and  subjective  responses  to  the  decision  support  system. 

FUTURE  RESEARCH 

While  tools  based  on  both  the  recognition-primed  decision  and  explanation-based  reasoning 
models  of  decision-making  are  included  in  the  DSS,  no  direct  connection  exists  between  the  two. 
Research  is  currently  being  conducted  to  extend  schema  theory  to  dynamic  decision-making  situa¬ 
tions.  This  involves  developing  and  testing  a  hybrid  model  of  cognitive  behavior  in  decision-making 
to  incorporate  both  types  of  knowledge,  i.e.,  feature  matching  and  story  generation,  as  elements  of  the 
same  schema  model  of  naturalistic  decision-making  (Smith  &  Marshall,  in  press).  Schema  theory,  as 
described  by  these  authors,  offers  a  context  for  integrating  these  two  models  that  have  typically  been 
viewed  as  separate  entities. 
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